1. Field of the Invention
This invention relates to computer systems and, more particularly, to systems for rapidly and accurately debugging computer systems.
2. History of the Prior Art
It is often desirable to detect errors in a computer process or a computer system. To accomplish this, one typically tests the behavior of the system being designed (the "test system") against models of a system (the "reference system") which are known to function correctly. If the behaviors of the test and reference systems agree, the test system is presumed to be functioning correctly. If the behaviors of the systems differ, an error has been detected.
There are many ways to compare the behavior of two systems. One is to cause each system to generate a stream of internal or external events such as procedure calls, state transitions, or bus signals, and compare the two streams. However, some event streams may be too difficult to capture to make comparison of the event streams practical. Other event streams may be too expensive to capture. Yet other event streams may be too coarse-grained to provide useful localization of the errors. In some event streams, the events may depend too much on the implementations of the systems and thus be incomparable.
Another comparison technique is to run both systems, record the state of the systems while running, and then compare the state of the systems. State comparison may suffer the same kinds of problems as the comparison of events streams. Some state may be expensive or impossible to access or may depend too much on implementation details of the two systems.
Different methods may be used to accomplish comparison of either state or events. Traditionally, where the reference system is informal or is a non-executable, the method has been an ad hoc search for various test state or events that are inconsistent with state or events produced by the reference. This method, of course, offers significant chance to miss important problems which may reside in the test system.
Where the reference is an executable specification, it would be desirable to be able to simply run the two systems simultaneously and compare the state at any instant to detect differences. The problems delineated above make this more difficult.
The exact comparison of system state after each step is not generally possible. First, the test system and reference system necessarily differ in some manner (such as function, implementation, or performance) so that the total state of the systems is not comparable at all times. With some systems, the total state may never be comparable. For example, it is often desirable to compare the state of a test system which is significantly different that the reference system as when porting an application program from one type of processor running a first operating system to another type of processor running a second operating system.
What is necessary is that the test system produce the same final results as the reference system. Since the processors and operating systems differ, the operations being performed at any instant in running the two systems (and thus the state) will probably differ.
Second, it may be prohibitively expensive to repeatedly compare the state after each operation by each system. In order to compare the state of two systems, all of memory (possibly including second level storage) and all of the registers of the two systems must be compared to see if they are the same. The sheer size of the state comparison makes this expensive.
Finally, some state information may be unavailable, for example state in unreadable processor registers or state that is represented implicitly in one of the systems. Therefore, the comparison must often be selective.
Even though the comparison of state at all points during execution is difficult, it is possible to select points at which the results of the test and reference system should be the same. At these points, a comparison of state may be taken. However, to date no automated or rapid method for comparing systems has been devised.
It is desirable to provide a method and apparatus for rapidly detecting errors in a test system.
More particularly, it is desirable to provide a method and apparatus for detecting errors in a computer process or a computer system.